[ci] Run MAUI R2R tests with XHarness - #12666
Conversation
Use Arcade's XHarness Android work items so transient device installation failures get the standard reboot and retry behavior while retaining the existing app launch smoke checks and diagnostics. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
The new Helix MSBuild project currently allows XHarnessApkToTest to run without validating AndroidPackageName, which can lead to confusing failures and should be gated/validated before merge.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review tier: Lite
Findings: 1
New issues introduced by this change (2)
| Severity | Finding |
|---|---|
 |
build-tools/automation/MauiR2RHelix.proj — ❌ error — XHarnessApkToTest is enabled when WorkItemApk is set, but AndroidPackageName can… |
 |
build-tools/automation/MauiR2RHelix.proj — Set-Content/Add-Content calls don’t specify an encoding. On Windows… |
What changed in this PR
This PR updates the MAUI R2R Helix submission flow in dotnet/android CI to use Arcade/Helix’s XHarnessApkToTest integration (instead of a bespoke adb-driven work-item script), aiming to improve resiliency on device install failures by leveraging XHarness’s retry/reboot behavior while preserving the existing smoke-test behavior and diagnostics uploads.
Changes:
- Adds an
eng/Versions.propspin for the XHarness CLI version intended to match MAUI’s usage. - Updates the MAUI R2R Helix matrix template to stage only the signed APK and submit Helix work items using
WorkItemApk/AndroidPackageNameenv vars, with a longer work-item timeout. - Switches
MauiR2RHelix.projfromHelixWorkItemtoXHarnessApkToTestand embeds PowerShell-based custom commands for install + monkey launch +pidofsmoke check + log capture/uninstall.
| File | Description |
|---|---|
| eng/Versions.props | Introduces a pinned XHarness CLI version property. |
| build-tools/automation/yaml-templates/run-maui-r2r-helix-matrix.yaml | Removes the custom adb/payload script generation and submits Helix work items via WorkItemApk + AndroidPackageName, increasing timeout. |
| build-tools/automation/MauiR2RHelix.proj | Uses XHarnessApkToTest with embedded custom commands to install/launch/validate/log/uninstall the APK. |
| <Command>$(WorkItemCommand)</Command> | ||
| <Timeout Condition="'$(WorkItemTimeout)' != ''">$(WorkItemTimeout)</Timeout> | ||
| </HelixWorkItem> | ||
| <XHarnessApkToTest Include="$(HelixWorkItemName)" Condition="'$(WorkItemApk)' != ''"> |
| "Scenario: $(HelixTestRunName)" | Set-Content -LiteralPath (Join-Path $output_directory "scenario-$(HelixConfiguration).txt") | ||
| "PackageName: $package_name" | Add-Content -LiteralPath (Join-Path $output_directory "scenario-$(HelixConfiguration).txt") | ||
| "APK: $app" | Add-Content -LiteralPath (Join-Path $output_directory "scenario-$(HelixConfiguration).txt") |
Resolve the central versions conflict by retaining the updated build task dependency and the XHarness CLI pin. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
|
/review |
|
✅ Android PR Reviewer completed successfully!
|
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
The XHarness migration preserves the existing launch/process verification and improves install resilience, but cleanup should be guaranteed after installation. CI is currently red in dotnet-android / MAUI Tests MAUI Integration; the runner did not permit access to the Azure DevOps details, so I could not determine whether that failure is caused by this change.
Generated by Android PR Reviewer for #12666 · gpt56 · 173.1 AIC · ⌖ 9.19 AIC · ⊞ 25.7K
Comment /review to run again
| xharness android adb -- -s "$deviceId" shell pidof "$package_name" | ||
| $pidExitCode = $LASTEXITCODE | ||
| xharness android adb -- -s "$deviceId" logcat -d -b all | Set-Content -LiteralPath (Join-Path $output_directory "logcat-$(HelixConfiguration).log") | ||
| xharness android uninstall --package-name "$package_name" --device-id "$deviceId" |
There was a problem hiding this comment.
🤖 💡 Resource management — Please put the post-install work in a try/finally and uninstall in the finally. As written, any terminating failure while collecting diagnostics, launching, sleeping, or writing log files bypasses this line and leaves the package installed; the explicit launch-failure cleanup only covers one path.
Rule: Clean up acquired resources on every exit path
Summary
adbwork-item script with Arcade'sXHarnessApkToTestintegrationxharness android installso package-manager failures receive XHarness's device reboot and install retry behaviormonkeylaunch,pidofsmoke check, and uploaded device/package/logcat diagnosticsContext
The existing script exits immediately on
INSTALL_FAILED_INSUFFICIENT_STORAGE, bypassing XHarness's recovery path. This contributed to the repeated Helix device failures tracked by dotnet/arcade#17483 and #12658.Validation
XHarnessApkToTestwork-item archive with the pinned Helix SDK11.0.0-prerelease.26428.1is available from the configured engineering feed